A small high speed local buffer known as a cache memory is often used to allow a high speed processor to operate at or near its maximum rate with a slower speed main memory. The cache saves the most frequently or most recently used contents of the main memory. When the processor accesses memory it is possible that the needed data or instruction is in the cache and that a longer main memory access can be avoided, thus increasing system performance. FIG. 1 illustrates a typical block diagram for a processor system using a cache memory.
Three major factors affect the performance improvement a cache can provide: the hit rate, the cache access time, and the miss penalty time. A hit occurs when an access can be completed out of the cache and an access to the main memory is not required. The hit rate is the number of accesses that can be completed in the cache divided by the total accesses. The cache access time is how long it takes to fetch data from the cache. Finally, the miss penalty is how long it takes to access data from the main memory and update the cache when there is not a cache hit.